Goto

Collaborating Authors

 safety tip


IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks

Lu, Xiaoya, Chen, Zeren, Hu, Xuhao, Zhou, Yijin, Zhang, Weichen, Liu, Dongrui, Sheng, Lu, Shao, Jing

arXiv.org Artificial Intelligence

Flawed planning from VLM-driven embodied agents poses significant safety hazards, hindering their deployment in real-world household tasks. However, existing static, non-interactive evaluation paradigms fail to adequately assess risks within these interactive environments, since they cannot simulate dynamic risks that emerge from an agent's actions and rely on unreliable post-hoc evaluations that ignore unsafe intermediate steps. To bridge this critical gap, we propose evaluating an agent's interactive safety: its ability to perceive emergent risks and execute mitigation steps in the correct procedural order. We thus present IS-Bench, the first multi-modal benchmark designed for interactive safety, featuring 161 challenging scenarios with 388 unique safety risks instantiated in a high-fidelity simulator. Crucially, it facilitates a novel process-oriented evaluation that verifies whether risk mitigation actions are performed before/after specific risk-prone steps. Extensive experiments on leading VLMs, including the GPT-4o and Gemini-2.5 series, reveal that current agents lack interactive safety awareness, and that while safety-aware Chain-of-Thought can improve performance, it often compromises task completion. By highlighting these critical limitations, IS-Bench provides a foundation for developing safer and more reliable embodied AI systems. Code and data are released under https://github.com/AI45Lab/IS-Bench.


RiskAwareBench: Towards Evaluating Physical Risk Awareness for High-level Planning of LLM-based Embodied Agents

Zhu, Zihao, Wu, Bingzhe, Zhang, Zhengyou, Wu, Baoyuan

arXiv.org Artificial Intelligence

The integration of large language models (LLMs) into robotics significantly enhances the capabilities of embodied agents in understanding and executing complex natural language instructions. However, the unmitigated deployment of LLM-based embodied systems in real-world environments may pose potential physical risks, such as property damage and personal injury. Existing security benchmarks for LLMs overlook risk awareness for LLM-based embodied agents. To address this gap, we propose RiskAwareBench, an automated framework designed to assess physical risks awareness in LLM-based embodied agents. RiskAwareBench consists of four modules: safety tips generation, risky scene generation, plan generation, and evaluation, enabling comprehensive risk assessment with minimal manual intervention. Utilizing this framework, we compile the PhysicalRisk dataset, encompassing diverse scenarios with associated safety tips, observations, and instructions. Extensive experiments reveal that most LLMs exhibit insufficient physical risk awareness, and baseline risk mitigation strategies yield limited enhancement, which emphasizes the urgency and cruciality of improving risk awareness in LLM-based embodied agents in the future.


Tinder alerting users to take safety precautions amid coronavirus outbreak

USATODAY - Tech Top Stories

Coronavirus concerns have now spread to dating apps. Tinder is urging its users to stay safe amid the COVID-19 outbreak with pop-up alerts. The notification randomly appears when swiping for matches with the headline, "your wellbeing is our #1 priority." The popular dating app also features a series of safety tips within the alert before linking users back to the World Health Organization (WHO) to learn more. The safety tips, aligned with WHO's recommendations, include avoiding touching your face and carrying hand sanitizer.


7 safety tips to remember for online dating

Los Angeles Times

When searching for Mr. or Ms. Right, online dating is now so widely accepted that personal safety sometimes gets overlooked. After all, most dates that start online end up in either a love connection or with the two of you going your separate ways. But it is easy to let your guard down or find yourself in an unsafe situation -- especially if Grey Goose swoops in. That's why it's important to have a few common-sense strategies that keep you safe, according to law enforcement experts and others. "Tell your friends where you're going and when you'll be home," said Helen Fisher, a senior research fellow with the famed Kinsey Institute and chief scientific advisor to Match.com.